® OJ 



Europaisches Patentamt 
European Patent Office 
Office europe n des brevets 




(11) Publication number : 0 467 527 A2 



EUROPEAN PATENT APPLICATION 



2i) Application number: 91305412.8 



© int. ci. 5 : G06F 15/38 



(3) Date of filing : 14.06.91 



(30) Priority: 15.06.90 JP 155570/90 



@) Date of publication of application 
22.01.92 Bulletin 92/04 



© Designated Contracting States : 
DE FR GB 



@ Applicant : International Business Machines 
Corporation 
Old Orchard Road 
Armonk, N.Y. 10504 (US) 



72) Inventor : Nagao, Katashi 
587-1 Kami-odanaka 
Nakahara-ku, Kawasaki-shI (JP) 
Inventor: Nomfyama, Hiroshi 
4-1-50 Saginuma 
Miyamae-ku, Kawasaki-shi (JP) 



(74) Representative : Killgren, Neil Arthur 
IBM United Kingdom Limited Intellectual 
Property Department Hursley Park 
Winchester Hampshire S021 2JN (GB) 



(S) Natural language apparatus and method and construction of a knowledge base for natural language 
analysis. 



(5?) A natural language analysis apparatus comprises : knowledge base means for storing first-type trees 
representing dependencies among words in sentences, and second-type trees representing taxonym 
relationships of words ; table means responsive to entry of a word to output ID data of said first-type tree 
in which said word appears, node location data of said word in said first-type tree, and to output ID data 
of said second-type tree in which said word is contained as a hyponym ; means for judging the 
structural ambiguity of an incoming sentence ; means for extracting a candidate pair of modifier and 
modifiee for each possible dependency for a sentence judged to be ambiguous structurally ; means for 
entering words comprising each said pair into said table means and for determining, on the basis of the 
output data, a path including said words at opposite ends and including some of the words appearing in 
the first-type tree ; means for calculating a path distance for each said pair ; and means for determining 
a most preferable dependency on the basis of said path distance calculated for each. 
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The present invention relates to an apparatus and a method for resolving structural ambiguities in sent- 
ences of a natural, and to a method of constructing a knowledge base for resolving such structural ambiguities. 

The term "words" here signifies nouns, verbs, adjectives, adverbs, and other semantic words, and excludes 
articles, prepositions, and other functional words. A semantic unit of successive words is also regarded as one 
5 word in some fields. For example, in documents related to computer technology the expression "virtual disk" 
is regarded as one word. The term "dependency" means a modifier-modifiee relationship among words. 

Resolution of structural ambiguities in sentences is a difficult problem for natural language processing sys- 
tems. An example of the problem is provided by prepositional phrase attachment ambiguities. The sentence 
"A user can log on the system with a password" is ambiguous as to whether the prepositional phrase "with a 
to password" is attached adverbially to the verb "log on," or as a postmodifier to the noun phrase "the system." 

Methods have been proposed for resolving structural ambiguities of sentences on the basis of sematic and 
functional information on words, phrases, and other consituent elements. One such method is theoretically 
based on the case grammar disclosed in an article entitled "Toward a modern theory of case" by Charles J. 
Fillmore on pp. 361-375 of "Modern Studies in English," published in 1969 by Prentice-Hall. The functions of 
15 the constituent elements of a sentence for a predicate are called, cases, and semantic case functions are speci- 
cally called semantic cases (see attached Table 1 ). 

In case grammar, each constituent element of a sentence is called a case element, and the adequacy of 
a sentence is evaluated by matching the cases and the case elements. Taking the above-indicated sentence 
as an example, the term "log on" is a predicate, while "a user" functions as an agent, "the system" as an object, 
20 and "a password" as an instrument. Each verb is assigned to a framework called a case frame in which the 
case of each verb and the constraint conditions of case elements with respect to the verb are defined. Any input 
outside the definition is rejected as being semantically inadequate. In practical language usage, however, the 
boundary between semantically acceptable and non- acceptable sentences is a delicate one, and this also 
depends on the context. For example, in the sentence "My car drinks gasoline," if the predicate "drink" only 
25 accepts a word indicative of a human (a word having the semantic attribute HUM) as its agent, the term "car" 
is rejected. However, if "car" is considered to be used metaphorically, it is accepted. Thus, in a case grammar 
system that uses attribute values can easily construct knowledge but is limited in application. 

Japanese Published Unexamined Patent Application 63- 91776 discloses a method of using statistical 
information on the frequency of words to calculate the degree of preference of syntactic analysis trees for sol- 
30 ving structural ambiguities. The method is described below. 

1. Multiple analysis trees are produced from an input sentence, and an acceptable one is selected from 
among them. However, making multiple parse trees can be difficult and time-consuming. Futhermore, the 
method uses information on words that are not closely related to the ambiguities. 

2. The statistical frequency of co-occurrence relationships between words is used to solve ambiguities. 
35 Therefore, individual exceptions cannot be dealt with. For example, when an ambiguity exists as to whether 

a certain word A modifies word B or word C, the method does not consider that although it is statistically 
usual for A to modify B, in a certain particular sentence it modifies C. Further, since the method requires 
sufficiently formalised data (for example, registration of "virtual machine" as "machine is virtual"), collecting 
data is costly in terms of processing time. 

40 3. Natural languages generally comprise an enormous number of words. Therefore, in order to extend 

coverage range, the method abstracts words to define a category called a semantic marker. However, the 
semantic marker must be rearranged for a different field. For example, the term "department" is classified 
into the category of organisation in a certain field P, and knowledge on the attachments of "department" 
is absorbed into statistical information on co-occurrence relationships between the organisation category 

45 and another category. However, when the term "department" is classified into another category in a different 

field Q, the knowledge in field P is useless in field Q. It is costly in terms of processing time to re-abstract 
words and re-collect statistical information for each field. 

Structural ambiguity, which is the greatest bottleneck in analysis of natural language sentences, is caused 
by the presence of multiple modifier-modifiee relationships (dependencies) among words. Such structural 

so ambiguity cannot be solved by grammatical knowledge alone, but requires semantic processing. In practice, 
semantic processing in natural language processing involves both efficiently constructing a requisite large- 
scale knowledge, and efficiently using that knowledge. 

In accordance with the present invention, there is now provided apparatus for natural languag analysis, 
the apparatus comprising: knowledge base means for storing first-type trees representing dependencies among 

55 words in sentences, and second-type trees representing taxonym relationships of words; table means respon- 
sive to entry of a word to output ID data of said first-type tree in which said word appears, node location data 
of said word in said first-type tree, and to output ID data of said second-type tree in which said word is contained 
as a hyponym; means forjudging the structural ambiguity of an incoming sentence; means for extracting a can- 
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t l a Ic £ ^ ° f 8aCh POSSible de P ende "<* «* * sent nee judged to b ambiguous struc- 

t tn f"" 9 ^comprising each said pair int(J sajd ^ means gnd for detemm op 

basis of the output data, a path mcluding said words at opposite ends and including some of the words appearing 

Z^r^tenZr * ' T^T* " diStanCe f ° r eaCh Said pa * and means for determining a most 
preferable dependency on the basis of said path distance calculated for each 

The present invention therefore overcomes the structural ambiguity by preliminarily defining dependencies 
among words as background knowledge and referring to the background knowledge in order to select an ade- 
quate dependency among candidate dependencies. More specifically, when natural language analysing 
apparatus of he present .nvention obtains a structure including multiple attachment candidates as a result of 
syntactac analysis of a sentence, the degree of preference of dependencies among words in the sentence is 
also obta.ned based on the dependencies among the words stored as the background knowledge. The 
apparatus can therefore determine which attachment is more preferable 

Viewing the present invention from a second aspect, there is provided a system for natural language 
analys.s. the system comprising: knowledge base means for storing first-type trees representing dependencies 
among words in sentences, and second-type trees representing taxonym relationships of words; table means 
responsive to entry of a word to output ID data of said first-type tree in which said word appears, node location 
data of said word m said first-type tree, and ID data of said second-type tree in which said word is contained 
as a hyponym; means forjudging the structural ambiguity of an incoming sentence; means for extracting a can- 
Ada e pa.rof modrfierand modifiee for each possible dependency for a sentence judged to be ambiguous struc- 
turally; means for entenng words comprising each said pair into said table means and determining, on the basis 
of the output data, a path including said words at opposite ends and including some of the words appearing in 
the first-type tree; means for calculating a path distance for each said pain and means for determining a most 
preferable dependency on the basis of said path distance calculated for each. 

Advantageously, a natural language analysis system of the present invention can be used for sentence 
analysis in. for example, machine translation systems, and question-and-answer systems using natural lan- 
guages to output the most preferable syntactic tree in response to an incoming sentence that includes structural 
ambiguities by using knowledge on synonym relationships, taxonym relationships, and dependencies among 
words. Such a system can thus solve problems that cannot be solved by conventional grammar-based analysis! 
such as amb.gu.taes that can only be solved by the use of expert knowledge in a specific field or by referring 
30 to the contents of a preceding sentence. 

A common characteristic of conventional analysis methods is that information relating to a word in a sent- 
ence, such as attributes for semantic classification, is very abundant and that this information is heuristically 
determined by human beings. In contrast, information required for a natural language analysis method of the 
present .nvention is described relatively formally, and large-scale new background knowledge can be construc- 
Svention" 131 ' 03 " Semiautomatically; thus makin 9 it relatively easy to construct apparatus of the present 

Specifically, a natural language analysis system of the present invention resolves structural ambiguities 
by initially expressing knowledge in the form of a tree structure indicative of synonym relationships, taxonym 
relationships, and dependencies among words. When a sentence is entered, the system searches for depen- 
dencies among words defined by the background knowledge, using synonym relationships and taxonym rela- 
tionships. Subsequently, using a consistency condition obtained from the sentence and one obtained from the 
context, the system selects the most acceptable attachment and solves the ambiguity. The decided depen- 
dency structure is registered in the knowledge base as context dependency data. 

The system can comprise means for storing in said knowledge base means a first-type tree for the incoming 
sentence mcluding said most preferable dependency, and for renewing said table means responsively. The 
knowledge base means can separately store learned data and context data added by said means for deter- 
mining. The table means can be separately prepared for learned data and for context data. The means for cal- 
culating can calculate said distance, based on the number of dependencies included in the path. The first type 
tree can be provided with semantic case data for each dependency. 

In a preferred embodiment of the present invention, the means for calculating calculates said distance 
accord.ng to the consistency between the case relationship between a modifier and a candidate modifiee and 
the case relationship for the path. In another preferred embodiment of the present invention, the means for cal- 
culating can calculate said distance, on the basis of the consistency of co-occurrence of a word included in 
said incoming sentence and a word included in said first-type tree for th path. In yet another preferred embo- 
ss diment of the present invention, th means for calculating calculates said distanc . on the basis of the degree 
of consistency between the path and a first-type tree added by said means for determining. 

Preferably, said second-type tree is an "isa" tree having only two nodes corresponding to a hypemym and 
a hyponym. wherein said means for entering is responsive to an output of a hypemym of a word forming the 



35 



40 



45 



SO 



BNSOOCID: <EP__0*87S27A2_I_> 



EP 0 467 527 A2 



pair, to iterate search for an "isa" tree including said hypernym as a hyponym, thereby producing a chain of 
hypemyms. A synonym relationship is preferably represented by two "isa" trees. 

Viewing the present invention from a third aspect, there is provided, in a computer system including a 
knowledge base that stores first-type trees representing dependencies among words in sent nces and sec- 

5 ond-type trees representing taxonym relationships of words, and including a table responsive to entry of a word 
for outputting ID data of said first-type tree in which said word appears, node location data of said word in said 
first-type tree, and ID data of said second-type tree in which said word appears as a hyponym, a natural lan- 
guage analysis method comprising the steps of: (a) judging the structural ambiguity of an incoming sentence; 
(b) extracting a candidate pair of modifier and modifiee for each possible dependency as for a sentence judged 

10 to be structurally ambiguous; (c) entering words comprising each pair into said table means and determining, 
on the basis of the output data, a path that has said words at opposite ends and contains some of the words 
appearing in said first-type tree; (d) calculating a path distance for each pair; and (e) determining the most pref- 
erable dependency relationship, on the basis of said path distance calculated for each said pair. 

Preferably, the method further comprises the step of: (h) storing in said knowledge base a first-type tree 

15 for the incoming sentence including said most preferable dependency determined by said step (e) and renewing 
said table responsively. The knowledge base preferably stores learned data and context data added by said 
step (0 separately. Preferably, the table can separately prepared for learned data and context data. Step (d) 
preferably calculates said distance, on the basis of the number of dependencies included in the path. The first 
first-type tree is preferably provided with semantic case data for each dependency. 

20 In a preferred example of a method according to the present invention, step (d) calculates said distance 

according to the consistency between the case relationship of a modifier and a candidate modifiee and the case 
relationship for the path. In another preferred example step (d) calculates said distance according to the co- 
occurrence consistency of a word included in said input sentence and a word included in said first-type tree 
for the path. In still another preferred example, step (d) calculates said distance according to the degree of con- 

25 sistency between the path and a first-type tree added by said step (f). 

The second-type tree is preferably an "isa" tree having only two nodes corresponding to a hypernym and 
a hyponym, and wherein said step (c) is responsive to an output of a hypernym of a word forming the pair, to 
iterate search for an "isa* tree including said hypernym as a hyponym, thereby producing a chain of hypernyms. 
Preferably, a synonym relationship is represented by two "isa" trees. 

30 Viewing the present invention from a fourth aspect, there is provided a method for constructing a knowledge 

base for natural language analysis comprising the steps of: (a) preparing a knowledge base that stores trees 
representing dependencies among words in sentences; (b) determining the most preferable of the possible 
dependencies for an incoming sentence by using said knowledge base; and (c) storing in said knowledge base 
a tree for the incoming sentence that includes said most preferable dependency. Preferably, said knowledge 

35 base separately stores learned data and context data added by said step (c). 

Viewing the present invention from a fifth aspect, there is provided a method of constructing a knowledge 
base for natural language analysis comprising the steps of: (a) preparing a knowledge base for storing trees 
representing dependencies among words in sentences and preparing a table responsive to entry of a word for 
outputting ID data of a tree containing said word and node location data of said word in said tree; (b) determining 

40 the most preferable of the possible dependencies for an incoming sentence by using said knowledge base and 
said table; and (c) storing in said knowledge base a tree for the incoming sentence that includes said most pref- 
erable dependency and renewing said table responsively. Preferably, said table is separately prepared for lear- 
ned data and for context data. 

An embodiment of the present invention will now be described with reference to the accompanying draw- 

45 ings in which: 

Figure 1 is an explanatory view of an arrangement of a natural language analysing system according to 
the invention; 

Figure 2 is an explanatory view of a phrase structure including ambiguities; 

Figure 3 is an explanatory view of a dependency structure including ambiguities; 
so Figure 4 is an explanatory view of possible dependency candidates; 

Figure 5 is an explanatory view of an example of phrase structure; 

Figure 6 is an explanatory view of an xample of dependency structure; 

Figure 7 is an explanatory view of dependencies and semantic cases; 

Figure 8 is an explanatory view of a taxonym relationship; 
55 Figure 9 is an explanatory view of a synonym relationship; 

Figure 10 is an explanatory view of a path; 

Figure 1 1 is an explanatory view of the node location of a word on a dependency structure tree; 
Figure 12 is an explanatory vi w of an "isa" tree; 
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Figure 13 is an explanatory view of a transition of a dependency; 
Figure 14 is an explanatory view of a path from word c to word a- 
Figur 15 is an explanatory view of the dependency between "keep" and "virtual disk"- 
Figure 16 is an explanatory view of a path from "virtual disk" to "keep" • 
Figure 1 7 is an explanatory view of a co-occurrence relationship; 
Figure 18 is an explanatory view of the dependency structure in a knowledge base; 
Figure 19 is an explanatory view of a co-occurrence relationship in the knowledge base- 
Figure 20 is an explanatory view of the taxonym relationship in the knowledge base; ' 
Figure 21 is an explanatory view of the co-occurrence relationship between path and a sentence- 
Figure 22 is an explanatory view of a dependency structure of context data; and 
Figure 23 is an explanatory view of a path in the context data. 
of1h^resennnvtnS ram * * COmputersystem for ^P'ementing the natural language analysing system 

firSt 10 F i 9Ure 2 i' 80 eXamp ' e ° f 3 com P uter svs tem for implementing a natural language analysis 
SSSEi 1 PreS ?? °" com P rises a Processor connected to a direct access data storage device 
(DASD) and a visual d.splay terminal having a keyboard. In use. the DASD stores a computer program for con- 
figuring the computer system as a natural language analysis of the present invention. A user can operate the 
analysis system via the visual display terminal. "t^aie me 

Elements of the analysis system of the present invention will now be described with reference to Figure 1 

2JSi;^?II!^r2J5i2r!r ,S * 1 ^ M ^ ,n to *" En9liSh ,an9Uage - H ° Wever> * Wi " be a PP re <*** •* the present 

invention not limited to any specific language. 

SYNTACTIC ANALYSER 



20 



sanlr** ^22 T yS ?K r6C f V6S 3 SentenCe and ° Utputs a s y ntactic sfructure involvi "9 ambiguities. The 
sentence "VM/SP keeps the information on the virtual disk" is syntactically analysed into a phrase structure 
involving attachment amb.guities. as shown in Figure 2. Syntactic analysis technology is not involved in the 
present invention, and its explanation is omitted. 

30 DEPENDENCY STRUCTURE ANALYSER 

This comprises a dependency structure builder, a dependency extractor, a dependency selector, and a 
dependency structure transformer. 

The dependency structure builder converts a phrase sbucture into a dependency structure explicitly indi- 
cating dependencies between words, as shown in Figure 3. The phrase structure attachment ambiguities are 
expressed as ambiguities in dependencies among words. The dependencies are provided with labels corre- 
sponding to semantic cases. These labels are determined by referring to the grammatical word sequence and 
prepositions, and are expressed as a candidate list of possible semantic cases 

The dependency extractor extracts ambiguous dependencies from the created dependency structure as 
shown in F.gure. 4. They are expressed as multiple possible candidate dependencies for one ambiguity 

The .dependency selector searches for relationships corresponding to possible dependency candidatesin 
tiie background knowledge. When relationships are found for two or more candidates, the most preferable rela- 
tionship is determined by using constraint conditions. This is explained later 

The dependency structure transformer selects the most likely dependency for each ambiguity and accord- 
in£y fransforms the dependency structure to resolve the structural ambiguity. In this case, the semantic case 
attached to the dependency is also determined uniquely. The output of the dependency structure analyser is 
ttie dependency structure of a sentence in which every ambiguity has been resolved. The determined depen- 
dency w.ll be a constraint for analysis of subsequent sentences, and is therefore registered in the knowledge 
base as context dependency data. 

m J" aC l° r iT e ^ L PreS6nt invention ' natural 'anguage processing system comprises a semi- auto- 
matically bu.lt knowledge base and a mechanism for selecting the best dependency by using the knowledge 

base. These are explained below. 

CONSTRUCTION OF A KNOWLEDGE BASE 

From collected information concerning words such as terminology commentary, the system extracts, rela- 
tionships between a certain word and another word, nam ly, their synonym relationships, taxonym relation- 
ships, and dependencies. These relationships form the knowledge base. 
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The system expresses an item of knowledg in the form of a tre structure, for the following reasons: 

1. The tre structure can be made semiautomatically by analysing a sentence. 

2. It is suitable for expressing taxonym relationships and depend ncies. A synonym relationship is con- 
verted into two taxonym relationships, as explained later. 

5 3. Inference for dependency structure analysis is basically a process of traversing up a branch of a tree 

structure. The knowledge base contains tree structures indicative of dependency structures obtained from 
a sentence and taxonym relationships obtained by converting the dependency structures. In other words, 
it is a group of tree structures. 

In conventional natural language processing systems, in which necessary information for sentence 
10 analysis is not formalised, frames or other means capable of expressing substantially anything have been used 
successfully. However, such frame expression is difficult to construct systematically, and largely relies upon a 
human effort Therefore, increasing the scale of the knowledge base is very laborious. 

The knowledge base of the present invention can be built in a bottom-up manner; this makes it easy to 
increase the scale, and also corresponds to the nature of the problem of structural ambiguities. Naturally, 
15 knowledge must be acquired beforehand by learning. The data in the knowledge base is generated by analysing 
commentaries on words, creating dependency structures, and then converting them. The process is described 
below. 

1. The syntactic analyser creates phrase structures from a sentence, and converts them into dependency 
structures that define attachment relationships among words. In the learning step, a person determines 

20 ambiguous attachment relationships, and specifies a structure. For example, from the sentence "Operating 

system stores files on the disk," the phrase structure shown in Figure 5 is created. It is converted into the 
dependency structure shown in Figure 6. This is done by the dependency structure builder of the depen- 
dency structure analyser. 

2. As shown in Figure 7, semantic case labels (agent, patient, location) are attached as attributes to links 
25 indicating dependencies among words. This behaves as a constraint condition for use in removing an 

ambiguity. These labels are unambiguously determined by a person in the learning step after their candi- 
dates have been attached by the dependency structure builder. 

3. The dependency structure, obtained by sentences indicating a taxonym (hypernym/hyponym) relation- 
ship and a synonym relationship between words such as "A is a B," "A is a synonym for B, M and so on, is 

30 converted into a structure in which A and B are connected by a link labelled with "isa." This structure is 

called the "isa" tree, and examples are shown in Figures 8 and 9. 

SELECTION OF THE MOST PREFERABLE DEPENDENCY 

35 In order to select the most preferable dependency, the system employs a method of (1) searching paths 

corresponding to respective dependencies (path search) in the knowledge base, and (2) calculating values, 
called dependency distances for respective paths, on the basis of constraint conditions (distance calculation). 
The system then selects the dependency corresponding to the path having the shortest dependency distance 
as the most preferable dependency. This is done by the dependency selector of the dependency structure anal- 

40 yser. The path search first limits the search space in the enormous amount of knowledge by using co-occurr- 
ence between words. The probability of occurrence of a single word in a natural language is very small, and 
thus very little knowledge is needed for two words actually co-occurring. As a result, those words subject to 
distance calculation, which create the heaviest calculating load, are very few. This results in a very efficient 
search. The path search and the distance calculation are described below. 

45 

1. Path Search 

A path between two words incorporates chains of synonyms and hypernyms starting from them and at least 
one dependency between the words at the ends of the chains. In other words, a path is a route between words 
so if a knowledge base is regarded as a graph with words at its nodes. For example, the path between the words 
"keep" and "virtual disk" is shown in Figure 10. 

The following algorithm has been developed in order to search for paths in a knowledge base. It uses the 
ind x table shown in Table 2. 

In the table, the symbol tx denotes a pointer of a tree in which the word appears, and values in parentheses 
55 indicate the node location of the word in th tree (see Figure 11). 

Labels are always affixed to "isa" or other branches as attributes of hyponym nodes; therefore, pointers in 
the column of "isa tree" indicate a tree in which th word appears at a lower level of th "isa" branch. It is found 
from the table (Table 2) that word a is located in position (0) of the "isa" tree tO, and word b is located in position 
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0 of the same tree to, as shown in Figure 12 

b is found. As a resu.t, synonym or hypem^cLs Z word a aTe created Syn ° nym °' * ™* 
cr^tr.r*' 16 ' 8 tha \ P fe 3 5 yP6mym f ° r Q> and that Q is a h yP em y m for R- this case, two "isa" trees are 

it is ^dtSSlTZ of *e"? a "-? eS indude ^ s "w° words: the hypernym and the hyponym. Therefore. 

information on the locabon of the hyponym in the tree is also indispensable nyponym. that 

Subsequently, in order to search for a path between two words, itis necessary to check whether any deoen 
dency ,s present between words, one of which appears in one synonym/hypernym ctJE2^«^3E 
appears ,n the other chain. This means checking whether a set of depend ^SSS^S^. 

° f the chai " s and a set of dependency trees containing a word contained inToSer c^have 

two woras in the tree. In the dependency trees, the node locations of the two words in the trees are used to 
check whether any dependency between words or any transition between them exists. H^. when a first woS 
3 r t^coTwor md,reCUy thrOU9h 8 ~* lhe — " ° f de ^ de ^ be JeenteTt 
mimZ w XamP ' e ' f ° r thS dependencv in whi «=h word d modifies word b. the position (0) of b and the position 

In othVrwo^ T 7 f! CtUre ^ {M0(10) ' t110(010 » "-"""8 d reveal <"* an ancestoTo d 
^TE£! T h a ^ nsition <* the dependency exists between b and d (see Figure 13? 

The 22! ?! T W . " "! de 3 iS 30 anCeStor ° f node b ' the route fram b * • is determined uniquely 

Therefore, discovery of a dependency is deemed to be equivalent to checking a positional ml JSnshto Tto 
presence or absence of a path between words can be found by using the "isa" trees to obiai ' JfttSl 

- ^ the " ° btainin9 e ' ement indUded COmm ° nly S6tS ° f dependen^sire trees 
SSXX ^oaZo h n J he H Ch h ai ? S ' 3nd bV Subse < uent * -P^"9 *. positional rSLtSS^JS 
SeTn^aa^c^^^ 

2. Dependency Distance Calculation 

s^sisrni ssrr :r set* - <d — — * - - - 

Constraint conditions are classifiable into three categories. The first is the condition constraint as to whether 

!7S3S^£!^ a bran ? of a dependency in a path co,Tesponds £^?££Z 

Ln 3 ™ p attachment (whether a certain word depends on a certain predicate as a subject or an object 

Tn FiourT s °^h amP !' aSSU ^ 3tthe P3th Sh ° Wn in R 9 ure 16 has been obta "^ for the dependenishS 
in Figure 15 of me sentence "VM/SP keeps the infom,ation on the virtual disk." The grammatical ie fa case 

case between store and disk. Here, the case consistency between the dependency and the oath holds since 
SdfbXee^ 

n M £l ^ dependency and the path, then the value of case consistency of the path is 1 ; otherwise ft 
°- ln th,s example, the value of case consistency of the path is 1 oinerwise, it 

the rllayonsh^t!! 90 ^ * T™* " ^ occurrence consistency, which is a constraint regarding 

o?r22l n P 5 ♦ words co-occurring in the same sentence. For example, when a certain word depends 

syYonTXpemyr 38 ^ MM ' ** ° f the ShOUld ba a s ^ worto! ■ £ 

thatlh^T T«T P,e : " VM/ ? P " fe the SUbj6Ct 0f " keep -" as shown in R 9 ure 17 - 'n contrast, assuming 
that the path of F,gure 16has been obtained from a dependency structure tree in the knowledge base as shown 
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in Figure 18, co-occurrence is found as shown in Figure 19. and it is also found that "operating system" is an 
agent of "store.- Further, if a taxonym relationship between "VM/SP" and "operating system" as shown in Figure 
20 is defined as the knowledge, it is found that the co-occurrence consistency of words holds between the path 
and the sentence, as shown in Figure 21. Here, since the grammatical case "subject" can have the semantic 
case "agent," case consistency also holds. In this fashion, the number of co- occurrence consistencies (con- 
current consistencies of words and cases) is the value of the co-occurrence consistency. In this example, the 
value of the co-occurrence consistency of the path is 1 (it is assumed that co-occurrence consistency for cases 
other than the subject does not hold). 

The third category is context consistency. If dependencies between words appearing in a path already exist 
in previous sentences, the dependencies are considered to be firmly supported by the context, and the depen- 
dency distance on the path becomes dose. 

For example, if the sentence "In VM/SP, the data is stored in the storage device" comes before the above 
sentence, the.n the dependency structure shown in Figure 22 is stored as the context data of the knowledge 
base (the object herein referred to is not. a semantic case but a grammatical case indicative of an object). If a 
path is sought between "store" and ""disk" of the dependency "store disk" appearing in the path, using 
synonym/taxonym relationships and context dependencies of the knowledge base/then the path shown in Fig- 
ure 23 is found, and it is found that the dependency between "store" and "disk" is defined in the context. Thus 
the number of dependencies contained in the path of Figure 16 and defined in the context is the value of context 
consistency. In this example, since one dependency is contained in the path, the value of context consistency 
of the path is 1. 

The value of dependency distance is calculated by using the values of the foregoing constraints and the 
number of dependencies contained in the path. More specifically, it is computed from the following formula: 

number of context consistency 

dependency = dependencies + value distance 

distance (case consistency (co-occurrence 

value +1) x consistency value +1) 

This formula assumes that case and co-occurrence consistency affect the entire path, but that context con- 
sistency affects each dependency included in the path. Here, n is a real number in the range 0 < n < 1, and is 
a heuristic parameter that represents the degree of unimportance of the context. The dependency distance in 
the above example is 0.125 because the number of dependencies is 1, the value of case consistency is 1, that 
of co-occurrence consistency is 1, and that of context consistency is 1 (n is defined as 0.5). 

REGISTRATION IN KNOWLEDGE BASE 

The dependency structure that has been determined to be most preferable is registered in the knowledge 
base and is used for resolving structural ambiguities of subsequently input data. Since the result of the decision 
greatly depends on the context, it is preferable to register the result independently as context dependency data 
in order to distinguish it from learned data (see Figure 1). More specifically, a knowledge base that stores infor- 
mation on the dependency structure and the semantic case, as shown in Figure 7, and the index table in the 
right half of Table 2 are prepared for context dependency data for each field. When the most preferable depen- 
dency has been determined, corresponding data is added to the knowledge base and to the index table. Dup- 
licate registration may be prevented by referring to a previously registered dependency. 

Thus, knowledge can be increased automatically. In a strict sense, the method is not fully automatic, since 
human intervention is needed in some operations however, knowledge is increased at least semi-automatically. 

PRACTICAL EXAMPLES 

1 . Syntactic Analysis of Input Sentence and Conversion into Dependency Structure: 

Input sentence 1: 

In VM/SP, the data is stored on the storage device. (This sentence has no structural ambiguity.) 
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Phrase structure ((DECL (PP (PREp 





(NOUN* 




(PUNC 


(NP 


(DET 




(NOUN* 


(VERB 


"is" ( 


(VERB* 


"stored 1 


(PP 


(PREP 




(DET 




(NOUN* 


(PUNC 


".")) 



(AD J* "the" ("the" BS))) 



10 (VERB* "st-rt*.*^" (" st( 

on ) 



(AD J* "the" ("the" BS))) 
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Dependency structure 



3 store 



1 vm/sp : (CONDITION) 2 data : (PATIENT) 4 storage device : LOCATION) 
I ! 

word semantic case label (if multiple labels are present, 

all possible ones are indicated) Input sentence 2: VM/SP 
keeps the information on the virtual disk. 

(This sentence includes structural ambiguities.) 



20 



25 



Phrase -structure: ((DECL (NP (NOUN* 
(VERB* "keeps" ("keep" PS) ) 



r vm/sp ("vm/sp" SG))) 



(NP 



30 



(PUNC 



(DET 

(NOUN* 

(PP 

(DET 

(NOUN* 

'V')) 



(AD J* "the" ("the" BS))) 
"information" ("information" SG) ) 
(PREP "on") 

(ADJ* "the" ("the" BS))) 

"virtual disk" ("virtual disk" SG)))) 

0) 



(a question mark indicates another dependency candidate) 
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2 keep 



1 vm/sp : (AGENT) 3 information : (PATIENT) 



4 virtual disk : (LOCATION CONDITION) 1 (2 3) 

I 

modifiee candidate 

The list of modifiee candidates (2 3) represents that the word ("virtual disk") can be attached to word 2 
("keep") or word 3 ("information"). 
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2 keep 

5 



10 



15 



20 



35 



1 vm/sp : (AGENT) 3 information ; (PATIENT) 



4 virtual disk : (LOCATION CONDITION) ! (2 3) 
In the dependency tree, two dependency candidates "information" <- "virtual disk" and "keep" <- "virtual disk" 

P«ttM<T421 (tre. ID In the kno«l«o. base)) ((-WomaSonW «Mm,al disk- Ml s k-)0O) This pall. Is shown 



information 

j location 
disk <- virtual disk 
25 isa 

Number of dependencies in the path: 1 
Value of case consistency: 1 
Value of co-occurrence consistency: 0 
30 Value of context consistency: 0 
Dependency distance: 0.5 

Dependency distance of "information" <- "virtual disk" is 0 5 

Path 8 ^ iS d ° ne re9ardin9 the de P en <*ncy W <" "virtual disk/ 

Path. ((T425) (( keep" "store")) (("virtual disk" "disk")2)) 



This path is shown below: 



40 



45 



isa 

keep -> store 

I location 
disk <- virtual disk 
isa 



50 
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Number of dependencies in the path: 1 
Value of case consistency: 1 

This path is obtained from the following dependency structure: 

2 store 



1 operating system : AGENT 3 fi 



e : PATIENT 4 disk : LOCATION 
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List of co-occurring words in the sentence: ("vm/sp" . AGENT) ("information* . PATIENT)) List of co-occurring 
words in the path: (("operating system* . AGENT) ("file" . PATIENT)) Value of co-occurrence consistency: 1 
Context path: (((T426) ("store")) (("disk" "storage device")2)) This context path is shown below (this is obtained 
from the dependency of the preceding sentence): 

5 

store 

| location 
storage device <- disk 

10 

isa Value of context consistency: 1 Dependency 

distance: 0.125 

Dependency distance of "keep" <- "virtual disk" is 0.125. 
is In other words, the dependency "keep" <- "virtual disk" is found to be most preferable, and the dependency 
structure is modified as follows: 

2 keep 

20 



1 vm/sp : AGENT 3 information : PATIENT 4 virtual disk : LOCATION 

25 

EXPERIMENTAL RESULTS 

The ability of the present invention system to resolve prepositional attachment ambiguities, has been tested 
by using approximately 2,000 sentences, extracted from a computer manual. The result is shown below. The 
30 knowledge used here consists of the dependency structures extracted from about 20,000 definition sentences 
in the "IBM Dictionary of Computing." 



Total number of 
prepositional phrases 


Number of attachments correctly 
disambiguated by the system 


4290 


3569 


Success ratio 


4290/3569 x 100 = 83.2% 



The results show that the system is significantly effective. 
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Table 1. Examples of semantic 



case 



Semantic case 



10 
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Agent 

Patient 

Instrument 

Object 



Source 

Target 

Location 
Time 



Role 



Person who causes a certain action 
Person who experiences a certain event 
Cause of a certain event, or stimulation 
causing a certain reaction 
Object to be moved, object to be changed, 
and contents of consideration or other 
psychological movement 

Starting point for movement of an object, 
initial aspect of a change in a state 
Terminal point for movement of an object, 
terminal aspect of a change in a state 
Location and position of a certain event 
T±me at whi <* a certain event occurs 



20 



Table 2. Index tablt 



25 



Words 


isa trees 


Dependency trees 


a 
b 
c 
d 


t0(0) tlO(O) t22(0) 
t5(l) t52(0) t62(0) 
t2(0) tl5(0) t72(l) 
t8(l) t25(l) t82(0) 


tl01(0 1) tl50(l 0) 
t30(l) tll0(0) 
tlOlCl 1) t350(0 2 3) 
t40(l 0) tll0(0 1 0) 
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2. 



Apparatus for natural language analysis, the apparatus comprising: 

ences a^S^^niT 3 " 8 *** St ° rin9 flraMype **** re P resen «ng dependencies among words in sent- 
ences, and second-type trees representing taxonym relationships of words; . 

.«™ ^ S . anS , reSP ?? iVe « t0 6ntry ° f 3 WOrd to output ,D data of said »™« i" which said word 

ISTSE? T 43 ° f WOrd 831(1 firet '* pe and to out P ut data of said second-type 
tree in which said word is contained as a hyponym; 

means forjudging the structural ambiguity of an incoming sentence; 

sentenc^-d 2225 ' P " r ° f M ™ a " d m ° difiee for each P^'ble dependency for a 

sentence judged to be ambiguous structurally; 

the hJH^L T*?^ W ° rdS P^P*^ 63(50 «■« P 3 ^ into said table means and for determining, on 

aToeanno Si ? indUdin9 SaW 3t ° PPOsite ends and indudin 9 so "ie of the w^rds 
appeanng in the first-type tree; 

means for calculating a path distance for each said pain and 
for each. 63 " 8 f ° rdetenninin9 3 most Parable dependency on the basis of said path distance calculated 

ftrs^et 3 ! 25"? in 1 * *" ,h<r v™*™* means for storin 9 «n said knowledge base means a 
mlim^!/ ,nCO T', n9 SentenC8 inC,UdinQ S3id most Parable depend ncy determined by said 
means for determining and for renewing said table means responsively. 

H S » aS !l?Tu Cl ! im 2 Wher8in Mid knowledge base means separately stores learned data and 
context data added by said means for storing. 
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4. Apparatus as claimed in claim 2 wherein said table means is separately prepared for learned data and for 
context data. 

5. Apparatus as claimed in claim 1 wherein said means for calculating calculates said distance, based on 
5 the number of dependencies included in the path. 

6. Apparatus as claimed in claim 1 wherein said first-type tree is provided with semantic case data for each 
dependency. 

10 7. Apparatus as claimed in claim 6 wherein said means for calculating calculates said distance according to 
the consistency between the case relationship between a modifier and a candidate modifiee and the case 
relationship for the path. 

8. Apparatus as claimed in claim 1 wherein said means for calculating calculates said distance, on the basis 
15 of the consistency of co-occurrence of a word included in said incoming sentence and a word included in 

said first-type tree for the path. 

9. Apparatus as claimed in claim 1 wherein said means for calculating calculates said distance, on the basis 
of the degree of consistency between the path and a first-type tree, added by said means for determining. 

20 

10. Apparatus as claimed in claim 1 wherein said second-type tree is an "isa" tree having only two nodes cor- 
responding to a hypemym and a hyponym, and wherein said means for entering is responsive to an output 
of a hypernym of a word forming the pair, to iterate search for an "isa" tree including said hypemym as a 
hyponym, thereby producing a chain of hypemyms. 

25 

11. Apparatus as claimed in claim 1 wherein wherein a synonym relationship is represented by two "isa" trees. 

12. In a computer system including a knowledge base that stores first-type trees representing dependencies 
among words in sentences and second-type trees representing taxonym relationships of words, and 

30 including a table responsive to entry of a word for outputting ID data of said first-type tree in which said 

word appears, node location data of said word in said first-type tree, and ID data of said second-type tree 
in which said word appears as a hyponym, a natural language analysis method comprising the steps of: 

(a) judging the structural ambiguity of an incoming sentence; 

(b) extracting a candidate pair of modifier and modifiee for each possible dependency as for a sentence 
35 judged to be structurally ambiguous; 

(c) entering words comprising each pair into said table means and determining, on the basis of the output 
data, a path that has said words at opposite ends and contains some of the words appearing in said 
first-type tree; 

(d) calculating a path distance for each pain and 

40 (e) determining the most preferable dependency relationship, on the basis of said path distance calcu- 

lated for each said pair. 

13. A method of constructing a knowledge base for natural language analysis comprising the steps of: 

(a) preparing a knowledge base for storing trees representing dependencies among words in sentences 
45 and preparing a table responsive to entry of a word for outputting ID data of a tree containing said word 

and node location data of said word in said tree; 

(b) determining the most preferable of the possible dependencies for an incoming sentence by using 
said knowledge base and said table; and 

(c) storing in said knowledge base a tree for the incoming sentence that includes said most preferable 
50 dependency and renewing said table responsively. 
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(PARSER) 
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; dependency' 

! "STRUCTURE 
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DEPENDENCY 
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DEPENDENCY 
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DEPENDENCY ' 
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TRANSFORMER 1 
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DEPENDENCY/ 
TAXONYM/ 
SYNONYM DATA 



CONTEXT 

DEPENDENCY 
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disk 
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'CMS is an operating svstem " 

ng system. operating system 

j isa 
CMS 
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tk l Jsa tisa 
authorized program pr ivileged program 
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